Method and system for correcting the effect of non-uniform illumination

ABSTRACT

A system and a method are provided for correcting the effect of non-uniform illumination on a reproduced image of a 2-D object. An image of a reference card in an illuminated setting is captured by an image capturing device. A 2-D object is placed in the same illuminated setting and the image of the object is captured. Next, a smoothing function for the reference card image is generated. The smoothing function is used to generate a smoothed reference card image. The smoothed reference card image is processed to compute inverse correction values, which are mathematical inverse of the pixel values in the smoothed reference card image. A pixel-by-pixel correction is then applied to the captured image of the 2-D object using the inverse correction values.

BACKGROUND

The present invention relates generally to the field of imageprocessing.

Fine art reproduction has been an area of commercial interest.Two-dimensional (2-D) original artworks, such as paintings, drawings andposters, can be reproduced as printed reproduction by photographing theartworks under a lighting set-up, and printing the captured images onposter-size media. The faithfulness of the reproduction is affected byseveral factors, one of which is the illumination on the objects beingphotographed. When a 2-D object is photographed by a camera undernon-uniform illumination, the captured image of the object containsregions that are unnaturally brighter or darker than other regions. Inconventional photography, a lot of time is spent in trying to get thelighting uniform by manually adjusting the lighting in order to achieveuniform illumination. This technique is very time consuming, inaccurateand demanding. For very large, museum-size paintings, it is practicallyimpossible to achieve perfectly uniform illumination on the paintings.

There remains a need for a simple method of correcting the non-uniformillumination characteristic in reproduced images of 2-D objects withoutrequiring the illumination on the objects to be uniform.

SUMMARY

A system and a method are provided for correcting the effect ofnon-uniform illumination on a reproduced image of a 2-D object. An imageof a reference card in an illuminated setting is captured by an imagecapturing device. A 2-D object is placed in the same illuminated settingand the image of the object is captured. Next, a smoothing function forthe reference card image is generated. The smoothing function is used togenerate a smoothed reference card image. The smoothed reference cardimage is processed to compute inverse correction values, which aremathematical inverse of the pixel values in the smoothed reference cardimage. A pixel-by-pixel correction is then applied to the captured imageof the 2-D object using the inverse correction values.

The objects, feature and advantages of the present disclosure willbecome apparent from the detailed description when read in conjunctionwith the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows the basic steps of a method for correcting the effect ofnon-uniform illumination in accordance to an embodiment.

FIG. 2 shows a schematic diagram of a system for correcting the effectof non-uniform illumination on a 2-D object in accordance to anembodiment.

DETAILED DESCRIPTION

FIG. 1 illustrates a method for correcting the effect of non-uniformillumination on reproduced images of 2-D objects. The 2-D objectsinclude paintings, drawings, posters, other 2-D artworks, and documents.This method eliminates the need for uniform illumination. At step 10, a2-D object is placed in an illuminated setting and the object's image iscaptured by an image capturing device, e.g. a digital camera. At step20, a reference card is placed in the same illuminated setting and animage of the reference card is captured. Alternatively, steps 10 and 20could be reversed so that the image of the reference card can becaptured before capturing the image of the 2-D object in the sameilluminated setting. In one embodiment, the reference card is auniformly colored white surface that is the same size as the 2-D object.Examples of a reference card can include a stretched canvas, a posterboard, and a wall surface. A white surface or board has been found to beeffective as a reference card. As an example, if the 2-D object is alarge painting hung on a white wall, the white wall could be used as areference card.

In one embodiment, the method can be optimized by using a reference cardessentially having a perfectly smooth and uniform surface. This,however, is difficult to achieve in real imaging situations. Hence, inanother embodiment, the reference card may contain defects, smudgesand/or blemishes from constant handling and the materials used formaking the reference cards are often not very uniform. These blemishesin the reference card as well as photon noise and dark noise may producea noisy reference image. Thus, in this embodiment, it may beadvantageous to smooth out the captured reference card image. This isaccomplished by generating a smoothing function based on the referencecard image at step 30, then generating a smoothed reference card imageusing this smoothing function at step 40. One of the advantages ofhaving steps 30 and 40 is that it is not necessary to have a perfectlyuniform and smooth (i.e., defect free) reference card. The referencecard may have holes, missing sections or may be slightly smaller thanthe 2D image and the smoothing function will “fill in” the missingportion. Furthermore, the smoothed reference card image does not have tobe at the same resolution as the originally captured reference cardimage. This can dramatically reduce the capture time for acquiring thereference card image.

In one embodiment, the smoothing function is a polynomial function thatfits to the data of the captured reference card image, but any smoothingfunction can be used. A generic Nth order polynomial function that issuitable for smoothing three-channel reference card image is definedbelow: $\begin{bmatrix}r^{\prime} \\g^{\prime} \\b^{\prime}\end{bmatrix} = {{s( {x,y} )} = {{\sum\limits_{k = 0}^{({N - 1})}\quad{\sum\limits_{l = 0}^{({N - k - 1})}\quad{c_{l,k}x^{l}y^{k}}}} = {\sum\limits_{k = 0}^{({N - 1})}\quad{\sum\limits_{l = 0}^{({N - k - 1})}\quad{\begin{bmatrix}c_{l,k}^{r} \\c_{l,k}^{g} \\c_{l,k}^{b}\end{bmatrix}x^{l}y^{k}}}}}}$Here, $\begin{bmatrix}r^{\prime} \\g^{\prime} \\b^{\prime}\end{bmatrix}\quad$are the smoothed reflectance image R, G, B values, $\begin{bmatrix}c_{l,k}^{r} \\c_{l,k}^{g} \\c_{l,k}^{b}\end{bmatrix}\quad$are coefficients that define the polynomial function (determined usingthe captured reference card image), x, y are the image locations of apixel in the captured reference card image, and N is the order of thepolynomial.

To illustrate how the coefficients of the polynomial function aredetermined from the captured reference card image, the followingdefinitions are necessary. Let:

-   m=number of pixels of the captured object's image,-   n=number of pixels of the captured reference card image (n does not    have to be equal to m),-   r_(i), g_(i), b_(i)=RGB data for pixel i of the captured reference    card image,-   r′_(i), g′_(i), b′_(i)=RGB data for pixel i of the smoothed    reference card image obtained after applying polynomial surface    fitting,-   x_(i), y_(i)=location of pixel i in the captured reference card    image (for i=1, 2, 3 . . . n),-   x′_(i), y′_(i)=location of pixel i in the captured object's image    (for i=1, 2, 3 . . . m).

Furthermore, assume that a 5th order polynomial function is appropriatefor representing the lighting on the artwork. With these assumptions,the smoothing polynomial function can be written as:s(x,y)=c _(0,0) +c _(1,0) x+c _(2,0) x ² +c _(3.0) x ³ +c _(4,0) x ⁴ +c_(0,1) y+c _(1,1) xy+c _(2,1) x ² y+c _(3,1) x ³ y+c _(0,2) y ² +c_(1,2) xy ² +c _(2,2) x ² y ² +c _(0.3) y ³ +c _(1,3) xy ³ +c _(0,4) y ⁴The coefficients c_(0,0), c_(1,0) . . . c_(0,4) of the polynomial arecomputed from the known r, g, b values of each pixel and x and ylocation of each pixel. To compute the coefficients, the 5^(th) orderpolynomial equation can be written in matrix form as: $\begin{bmatrix}r_{1} & g_{1} & b_{1} \\r_{2} & g_{2} & b_{2} \\\vdots & \vdots & \vdots \\r_{n} & g_{n} & b_{n}\end{bmatrix} = {\begin{bmatrix}1 & x_{1} & x_{1}^{2} & \cdots & {x_{1}y_{1}^{3}} & y_{1}^{4} \\1 & x_{2} & x_{2}^{2} & \cdots & {x_{2}y_{2}^{3}} & y_{2}^{4} \\\vdots & \vdots & \vdots & ⋰ & \vdots & \vdots \\1 & x_{n} & x_{n}^{2} & \cdots & {x_{n}y_{n}^{3}} & y_{n}^{4}\end{bmatrix}\quad\begin{bmatrix}c_{0,0}^{r} & c_{0,0}^{g} & c_{0,0}^{b} \\c_{1,0}^{r} & c_{1,0}^{g} & c_{1,0}^{b} \\c_{2,0}^{r} & c_{2,0}^{g} & c_{2,0}^{b} \\\vdots & \vdots & \vdots \\c_{1,3}^{r} & c_{1,3}^{g} & c_{1,3}^{b} \\c_{0,4}^{r} & c_{0,4}^{g} & c_{0,4}^{b}\end{bmatrix}}$In this form, the standard pseudo-inverse can be used to solve for thematrix of polynomial coefficents. Using the computed coefficients, asmoothed reference card image can be generated using the polynomialfunction at the same resolution as the originally captured referencecard image.

Those skilled in the art will recognize that other techniques forsolving for the coefficients in the polynomial equation are possible.For example, QR decomposition and SVD decomposition are two otherinversion techniques that could be applied.

Referring again to FIG. 1, inverse correction values are computed fromthe smoothed reference card image at step 50. The inverse correctionvalues are the mathematical inverse of the pixel values in the smoothedreference card image. Let r_(a), g_(a), b_(a) be the pixel values of thecaptured object's image at some location and r′_(i), g′_(i), b′_(i) bethe pixel values of the smoothed reference card image at the samelocation. The inverse correction value for this location (m^(r), m^(g),m^(b)) is defined bym ^(r)=1/r′ _(i) m ^(g)=1/g′ _(i) m ^(b)=1/b′ _(i)

At step 60, a pixel-by-pixel correction is applied to the capturedobject's image using the inverse correction values computed from step50. In one embodiment, this correction is done by multiplying each pixelvalue of the captured object's image with the corresponding inversecorrection value as shown by the following expression:R _(a) =r _(a) ×m ^(r) G _(a) =g _(a) ×m ^(g) B _(a) =b _(a) ×m ^(b)

Where R_(a), G_(a), B_(a), is the red, green and blue pixel values ofthe captured object's image after correction at the same pixel location.

Referring to FIG. 2, an embodiment of a system for carrying out themethod described in FIG. 1 includes a reference card 1, a light source2, an image capturing device 3, and an image processor 4. The lightsource 2 may be a lamp or another display illuminant. It is preferredthat the light source does not produce specular highlights or harshshadows. Although only one light source is shown in FIG. 2, two or morelight sources may be arranged so as to project more balanced lightingonto the 2-D object. The image capturing device may be a camera, forexample, a digital camera. The captured images of the reference card andthe object are processed by the image processor 4 to correct the effectof non-uniform illumination. The image processor 4 contains a programstorage medium embodying a program that performs the steps of:generating a smoothing function for the reference image; generating asmoothed reference image using the smoothing function; computing inversecorrection values from the smoothed reference card image; and applyingpixel-by-pixel correction to the object's image using the inversecorrection values. The image processor 4 may be contained fully or inpart within the image capturing device 3 or may be completely separatefrom the image capturing device 3. It will be understood by one skilledin the art that the processing steps done by the image processor 4 maybe performed using hardware, e.g. specialized ASIC, software, or anycombination thereof.

After the illumination non-uniformity in the object's image iscorrected, the object's image may be color corrected and output to aprinter 5 or a display 6. It will be understood by one skilled in theart that other output devices are also possible.

It is intended that the embodiments contained in the above descriptionand shown in the accompanying drawings are illustrative and notlimiting. It will be clear to those skilled in the art thatmodifications may be made to the embodiments without departing from thescope the invention as defined by the appended claims.

1. A method for correcting the effect of non-uniform illumination on areproduced image of a two-dimensional object, said method comprising:capturing an image of a reference card in an illuminated setting usingan image capturing device; capturing an image of a two-dimensionalobject placed in the same illuminated setting; generating a smoothingfunction for the captured reference card image; using the smoothingfunction to generate a smoothed reference card image; computing inversecorrection values, which are mathematical inverse of the pixel values inthe smoothed reference card image; and applying a pixel-by-pixelcorrection to the captured image of the two-dimensional object using theinverse correction values.
 2. The method of claim 1, wherein thesmoothing function is a polynomial surface fitting of the capturedreference card image.
 3. The method of claim 2, wherein the polynomialsurface is represented by $\begin{bmatrix}r^{\prime} \\g^{\prime} \\b^{\prime}\end{bmatrix} = {{s( {x,y} )} = {{\sum\limits_{k = 0}^{({N - 1})}\quad{\sum\limits_{l = 0}^{({N - k - 1})}\quad{c_{l,k}x^{l}y^{k}}}} = {\sum\limits_{k = 0}^{({N - 1})}\quad{\sum\limits_{l = 0}^{({N - k - 1})}\quad{\begin{bmatrix}c_{l,k}^{r} \\c_{l,k}^{g} \\c_{l,k}^{b}\end{bmatrix}x^{l}y^{k}}}}}}$
 4. The method of claim 1, wherein thereference card is uniformly colored.
 5. The method of claim 4, whereinthe reference card is white.
 6. The method of claim 4, wherein thereference card has substantially the same size as the two-dimensionalobject.
 7. The method of claim 4, wherein the two dimensional object isa painting or another two-dimensional artwork.
 8. The method of claim 1,wherein the pixel-by-pixel correction comprises multiplying each pixelvalue in the captured image of the two-dimensional object with acorresponding inverse correction value.
 9. A program storage mediumreadable by a computer, said program storage medium embodies a programof instructions executable by said computer to perform a method forcorrecting the effect of non-uniform illumination on a reproduced imageof a two-dimensional object, said method comprising: receiving an inputdigital image of a reference card in an illuminated setting; receivingan input digital image of a two-dimensional object placed in the sameilluminated setting; generating a smoothing function for the image ofthe reference card; using the smoothing function to generate a smoothedreference card image; computing inverse correction values, which aremathematical inverse of the pixel values in the smoothed reference cardimage; and applying a pixel-by-pixel correction to the image of thetwo-dimensional object using the inverse correction values.
 10. Theprogram storage medium of claim 9, wherein the smoothing function is apolynomial surface fitting of the captured reference card imagerepresented by $\begin{bmatrix}r^{\prime} \\g^{\prime} \\b^{\prime}\end{bmatrix} = {{s( {x,y} )} = {{\sum\limits_{k = 0}^{({N - 1})}\quad{\sum\limits_{l = 0}^{({N - k - 1})}\quad{c_{l,k}x^{l}y^{k}}}} = {\sum\limits_{k = 0}^{({N - 1})}\quad{\sum\limits_{l = 0}^{({N - k - 1})}\quad{\begin{bmatrix}c_{l,k}^{r} \\c_{l,k}^{g} \\c_{l,k}^{b}\end{bmatrix}x^{l}y^{k}}}}}}$
 11. A system for generating uniformlyilluminated images of two-dimensional objects, said system comprising:means for illuminating a two-dimensional object; a reference card; meansfor capturing an image of the reference card and an image of thetwo-dimensional object; and means for processing the captured images,said means for processing being programmed to perform a methodcomprising: (a) generating a smoothing function for the capturedreference card image; (b) generating a smoothed reference card imageusing the smoothing function; (c) computing the mathematical inverse ofthe pixel values in the smoothed reference card image in order to obtaininverse correction values; and (d) applying a pixel-by-pixel correctionto the captured image of the two-dimensional object using the inversecorrection values.
 12. The system of claim 11 further comprising meansfor printing the image of the two-dimensional object afterpixel-by-pixel correction.
 13. A system for generating uniformlyilluminated images of two-dimensional objects, said system comprising:at least one light source for illuminating a two-dimensional object; areference card; an image capturing device operable to capture an imageof the reference card and an image of the two-dimensional object; and animage processor for processing the captured images, said image processorbeing programmed to perform a method comprising: (e) generating asmoothing function for the captured reference card image; (f) generatinga smoothed reference card image using the smoothing function; (g)computing the mathematical inverse of the pixel values in the smoothedreference card image in order to obtain inverse correction values; and(h) applying a pixel-by-pixel correction to the captured image of thetwo-dimensional object using the inverse correction values.
 14. Thesystem of claim 13 further comprising a printer operable to print theimage of the two-dimensional object after pixel-by-pixel correction.